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Abstract 



en 

^ ' We investigate different versions of the minority game, a toy model for 

agents buying and selling a commodity. The Hamming distance between 

(^ ■ the strategies used by agents to take decisions is introduced as an analytical 

CN ! tool to determine several properties of these models. The success rate of the 

Q^ I agents in an adaptive version of the game is compared with the rate from a 

G^ I stochastic version. It is shown numerically and analytically that the adaptive 

bJ!)' process is inefficient, increasing the success rate of the unused strategies while 

Q ■ decreasing the success rate of the strategies used by the agents. The agents 



do not do as well as if they were forced to use only one strategy permanently, 
cd"! A version of the game in which the agents strategies evolve is also analyzed 



using the notion of distance. The agents evolve into a state in which they 
are all using one strategy, which is again the state that yields the maximum 
success rate. 
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1 Introduction 

A market is a network of interactions between agents buying and selling 
a commodity ^. On the one hand, the scarcity or the abundance in the 
commodity determines its price, while on the other hand, the price of the 
commodity modifies the supply and the demand equilibrium. Consequently, 
the variation of the price of the commodity reflects the dynamic of the market. 

The internal dynamic of a market is the result of the unrelated actions of 
the agents buying and selling. At any one moment in time, agents are either 
in the group of sellers or buyers, and agents can expect to profit if they are 
in the smaller group. Less sellers implies a higher price and a higher profit 
for the suppliers while if the number of buyers is small, the sellers will be 
forced to offer low prices to sell the commodity. This is a rough picture of 
how the internal dynamic distributes the profits amongst the agents. 

Obviously, the agents need to analyze the market's history to decide on 
their future action. This history is included in the market prices or in the 
volume of the exchanges. Hence, every agent has a strategy that uses the 
past prices of a commodity to predict the future price. In the course of time, 
the agents have to modify their strategies or their confidence in their different 
strategies to adapt themselves to the market changes. The profit of an agent 
will be determined by the success rate of his strategy and the efficiency with 
which he adapts to market changes. 

The minority game model is a very simple model that mimics the internal 
dynamic of the exchange of one commodity. The agents are allowed to buy or 
to sell a commodity at each time step and those choosing the smaller group 
among buyers and sellers are rewarded a point. The record of which group 
won for the past time steps constitutes the history of the system. Agents 
strive to maximise the number of points they are awarded. No attempt is 
made to model any external factors that influence the market. 

In the model, the agents have strategies at their disposal to analyse the 
history of the system and to take a decision for the next time step. In this 
paper, we introduce the notion of Hamming distance between strategies as 
an analytical tool to study different versions of the minority game. 

In Sec. 0, the basic features of the minority game model are introduced 
with the notion of strategy and Hamming distance between strategies. In 
Sec. 1^, the success rate of the agents in an adaptive version of the model 
is investigated. The difference between the success rate of a strategy and 



the success rate of an agent is outlined. Another non-adaptive version of the 
model is also introduced to question the relevance of the adaptive process. It 
is shown that if agents are allowed to choose between strategies they do not 
do as well as if they were forced to use only one strategy permanently. This 
result is confirmed in Sec. ^ where an evolutionary version of the minority 
game model is investigated. It is shown that most of the agents evolve 
spontaneously towards a state in which they are only using one strategy. 

2 The model 

The models we consider are different versions of the bar-attendance prob- 
lem introduced by Arthur |0| and simplified into a minority game by Challet 
and Zhang [^, ^. Basic to the minority game is an odd number A^ of agents 
having to choose between two sides, or 1, at each time step. An agent wins 
if he falls into the minority side. The record of which side was the winning 
side for the last m time steps constitutes the history of the system. For a 
given m, there are 2"^ different histories. The 8 different histories for m = 3 
are listed in the first column of table 1. 

Every agent makes a decision for the next time step according to the 
history of the system. To be able to play, an agent must have a strategy that 
allows him to make a decision for any of the different histories. The second 
and the third columns of table 1 list two possible sets of decisions a and a', 
that we will call strategies. 

If we compare a and a' component by component, we see that for some 
histories they make the same prediction and for others they make opposite 
predictions. In this example, the decisions only differ when the history is 
000, Oil or 110. The normalized Hamming distance between strategies a 
and a', 

d=—yWi-a'i\ (1) 

is a measure of the difference between the two strategies. In the example of 
table 1, d = 3/8. It corresponds to a time average of the different predictions 
of the two strategies, assuming that all the histories are equally likely to 
occur. Under this condition, d is the probability that an agent who always 



uses strategy a will take the opposite decision to an agent who always uses 
strategy a' . 

By definition, the distance is a number ranging from to 1. For a given 
771, the different distances available are 

ci„ = — , forn = 0,l,2,..,2"^. (2) 

If we choose two strategies at random, 

^(^) = 22™(2-rf)!(2-(l-rf))! ^^^ 

is the probability that the distance between these two strategies is d. We 
refer to -P((i) as the distance distribution. This is a binomial distribution 
with 

3=5 (4) 

for the mean value of d. The variance of the distribution is o"^((i) = 2"*^™'+^). 
In what follows, we investigate different versions of the minority game, to 
show the relevance of the notion of distance as an analytical tool. 



3 Success rates 

We first investigate the model introduced by Challet and Zhang 0]. Each 
agent has a fixed set of s > 1 strategies at his disposal. These strategies are 
chosen at random from all the different possible strategies, multiple choices 
of the same strategy are allowed. At each time step, every agent chooses one 
of his strategies to predict which side will be the minority side. The winning 
agents are awarded a point, while the points held by the losing agents do not 
change. In addition to giving points to the agents, the strategies also earn 
points themselves. A strategy earns a point each time it has forecast the 
correct winning side. The points awarded to strategies, called virtual points, 
allow an agent to measure the efficiency of all his strategies, irrespective of 
whether they were used or not. This measure of the success rate of the 
different strategies helps the agents to choose their future strategies. 

We distinguish between a determinisitic and a stochastic version for this 
model, according to the way an agent chooses his playing strategy. In the 



former, the agents pick the strategy with the highest number of virtual points 
while in the latter, they simply choose a strategy at random. 

There are two different issues to consider, firstly, that each strategy pos- 
sesses an opposite. Consequently, half of the strategies are potentially win- 
ning and half of the strategies are potentially losing, irrespective of the num- 
ber of agents playing or of the history of the previous days. Secondly, that 
there are N = 2k + l agents and at most k of them can win at each time step. 
Hence the probability of picking a winning strategy at random in the strat- 
egy space is exactly equal to 1/2 but the average probability that an agent 
wins is at most equal to k/{2k + 1) < 1/2. These two considerations and the 
fact that an agent does not have access to all the strategies means that the 
success rate of an agent is different from the success rate of a strategy. The 
difference between those two success rates is a measure of the ability of an 
agent to choose a performing strategy. We will now investigate the average 
success rates of the agents and their strategies for both versions of the model. 

First of all, we choose a reference strategy: for the stochastic version, any 
strategy is suitable, but for the deterministic version, we choose the strategy 
with most virtual points 0. Then, we assign an integer rank to all the other 
strategies: for the stochastic version, the rank r is given to all the strategies 
at distance dr = r jT^ from the reference one. For the deterministic version, 
we give rank to the strategy with the most virtual points and rank r to 
the strategy with the (r-|-l) largest number of virtual points. Strategies 
at the same distance, or with the same number of virtual points are given 
the same rank. We will assume that a strategy that is performing well is not 
very different from the best strategy, that is, the one with the most virtual 
points. Hence, we assume that the strategies of rank r in the deterministic 
version are also the strategies at distance dr = r/2™ from the best one. 
The average number of strategies at rank r is simply P{dr). As a result of 
these assumptions, we can estimate Nr, the average number of agents using 
a strategy of rank r in the next round of decisions. In the stochastic model, 

j^istoch) ^ JVP(rf,). (5) 

In the deterministic version, an agent uses a strategy of the r rank only if 
he has no strategy of a lower rank. The average number of agents using the 
best strategy, A^g , is 



<"^=iv(l-(l-^)y (6) 

while the average number of agents using a strategy of rank r > 0, N^!^'^^\ is 

Ni'^'^ = NY.-u^,indr)r-\i-jzp{d,)r (?) 

i^Ql\S I). j^Q 

also equal to 

Ni'iet) ^ N{{i-j:p{d,)r - (1 -j:p{d,)n (s) 

i=o i=o 

The average distance to the reference strategy is equal to 

d=-Y.Nrdr, (9) 

^^ r=0 

with Nj. taking the form of Eq. (^ or (P) and @ for the stochastic and 
deterministic versions of the model respectively. Notice that d < 0.5, with 
the equality holding only for the stochastic version of the model. 

Suppose now that there are iV = 5 (A; = 2) agents playing, numbered 
from 1 to 5. dij is the distance between the strategies used by agents i and 
j. An agent choosing one side wins if at least k + 1 = 3 other agents choose 
the other side. The success rate of agent 1, t2j\ defined as his probability to 
win a point per time step, is equal to 



^al^ = dl2di3dudi5 + di2di3du{l - rfis) + ^12^13(1 - (ii4)c/i5 

+ (ii2(l - (ii3)(ii4(ii5 + (1 - dy2)dyidxd\h- (10) 

The first term on the right hand side is the probability that agent 1 chooses 
one side and the other 4 agents the other side; the 4 other terms on the right 
hand side are the probabilities that only one agent chooses the same side as 
agent 1. For any other combination of choices agent 1 would lose. This can 
be generalized to any number of agents so that the average success rate for 
an agent can be expressed as 



where we have averaged over all the distances. For the deterministic version 
of the model, d is a function of s, but for the stochastic version, 
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expl^l - i /n(27riV) + iV/n(^^)) (12) 



where we have put d = 1/2. For the stochastic version of the model, all 
the agents are identical and r„(^*"'^^^ is an estimation of the success rate of the 
agents. The numerical and analytical values oiTJfg°'^'^' are compared in Fig. la 
(two lower curves) for A^ = 1001 and m = 5. The agreement is excellent. 
Notice that the success rate of the agents is independent of s. 

For the deterministic version, Eq. ([TT| ) gives the success rate of the ref- 
erence agent, r ° , which goes to zero very fast as a function of s. According 
to our classification, the success rate of an agent of rank r is 

<, = (l-rf.X + ^r(l-r°,). (13) 

which is approximately equal to dr for s > 2. The average success rate of 
any agent in the deterministic version is then 



r-' = ^Y.NrK- (14) 



-(det) 

■ag j\j Z^^-r ■ ag- 



=0 

This expression is compared to numerical results in Fig. lb (two lower curves) 
for A^ = 1001 and m = 5. The agreement is qualitatively good. The quan- 
titative discrepancies probably originate from our assumption that there is 
only one strategy with the highest number of virtual points. In fact, we also 
showed that in this case, the success rate of this strategy is approximately 
0. Hence, most of the time there is more than one strategy with the highest 
number of virtual points. 

Finally, we consider the success rates of the strategies in both versions of 
the model. When a strategy is used, it is in competition with one strategy 
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from each of the other agents. Consequently, it has the same average success 
rate as an agent. On the other hand, when a strategy is not used, it earns 
points only if it can predict the winning side. In the stochastic version, 
each side is as likely to be the winning side. Moreover, there are as many 
strategies suggesting one side as strategies suggesting the other. Hence, the 
average success rate for a strategy in the stochastic version of the model, 

(stoch) . 
Tst , IS 



r, 



(stock) _ 0.5(S - 1) + Tag 



St — 

S 



(15) 



In Fig. la this analytical result is compared with numerical simulations of the 
system for A^ = 1001 and m = 5. There is qualitative agreement between 
the two curves. 

For the deterministic version, the strategies that are not used have a 
greater average success rate than 0.5: they are not used because they differ 
from what we called the best strategy, the one with the most virtual points. 
But this strategy loses most of the time, as its very low success rate suggests. 
The more different a strategy is from the one with largest number of vitual 
points, the more likely it is to win. We can express the success rate of the 
unused strategies r^" as 

The average success rate of a strategy for the deterministic version is then 
equal to 

(det) ^st 1-5 ~ J-j + T^ 
Tst = ^, (17) 

because at each time step, one strategy is used and s — 1 are not. The 
numerical and analytical results for r]^ ^ are also reported in Fig. lb (two 
upper curves) for the same set of values. The two curves are in qualitative 
agreement. 

If we compare in both models the success rate of the agents and the 
success rate of their strategies, the agents seem to make very poor choices. 
In a set of strategies of a certain average success rate, the agents always pick 
strategies with a success rate lower than the average. In the stochastic model 



this occurs because the unused strategies have a success rate of 0.5 and there 
are more unused strategies for greater s. For the deterministic version, the 
adaptive process is such that the agents try to concentrate around the same 
points in the strategy space. Hence, they are all adaptating in the same way, 
making the same predictions and taking the same decisions. This process 
is only restricted by the part of the phase space available to the agents. 
Consequently, as s increases their performence deteriorates. Conversely, their 
adaptive process increases the success rate of the unused strategies instead 
of increasing the success rates of the agents. Obviously the problem with the 
adaptive process is that every agent is using the same strategy to adapt. To 
improve the model, an evolutionary process allowing for selection of efficient 
adaptive processes is required. In the next section, we investigate a version 
of the minority game model in which the strategies of the agents are allowed 
to evolve. 

4 The minority game with evolving agents 

The second version of the minority game we investigate is a model intro- 
duced by Johnson et al. 0. Every agent i has one strategy at his disposal. 
Some agents might share the same strategy. At each time step, each agent 
follows the prediction of his strategy with a probability pj, where i denotes 
the agent number i = 1, ...,N. The agent takes the opposite decision with a 
probability l—pi- Each time an agent chooses the minority side, he wins and 
earns a point. Otherwise, he loses and loses a point. The worst performing 
agents are allowed to change their probability Pi before the next time step 
commences. An agent scoring less than a given lower value L{< 0) chooses a 
new probability pi at random from the range [pi — R/2,pi + R/2]. Reflective 
boundary conditions are used, but our results do not depend on the particu- 
lar choice of boundary conditions. The agent's score is then reset to 0. The 
free parameter R allows one to adjust the persistence with which an agent 
sticks to an unsuccessful approach. 

We define Q{p)dp as the probability that an agent picked at random 
has an associated probability from the range [p — dp/2,p + dp/2]. Hence, 
Q{p) is the probability distribution of the probabilities p. We estimate the 
probability distribution Q{p) using the notion of distance. 

When an agent chooses a new strategy he is said to mutate. As the new 



strategy is choosen from the range [pi — R/2,pi + R/2], the time evolution of 
Q{p) is given by 



Q{p,t + dt) = Q{p,t) - S{p,t)dt 

+ ^{S{p + Ap,t) + S{p-Ap,t)) (18) 

where S{p, t) is the probabihty that any agent with probabihty p will mutate 
at time t. This can be rewritten as 



dt 2 dp^ ^ ' 

with the normalization condition 

fdpQ{p) = l. (20) 

As Ap is a measure of the variation of the probability associated with an 
agent when this agent mutates, we write Ap ~ i?/4. If Atm{p) is the time 
between two mutations for an agent associated with a probability p, the 
probability of mutation of this agent is 1/Atm{p) per time step. Hence, 

on average. We call r(p) the success rate of an agent with associated proba- 
bility p, supposing that this success rate depends only on p. This means that 
t{p) < 1/2. On average, the number of points earned by an agent, C{p,t), 
evolves as 

C{p,t) = C{p,to) - (i - r(p)) (t - to). (22) 

Applying this equation to t — to = Atm{p), 

Atmip) = -J ^ (23) 
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because an agent mutates when C{p,t) = L{< 0) and, after mutation, his 
points are reset to C{p, to) = 0. Hence, 
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Eq. (|2^) is a diffusion equation for the probability distribution in the space of 
p. The diffusion coefficient, (r — 0.5)-R^/32L, is a function of the position p. 
The normahzation condition of Eq. ([20| ) corresponds to a mass conservation 
law for Q{p). Integrating Eq. (El) with respect to p gives 



^(n 



dp VV2 
which can be rewritten as 



■(P)) Qip)) = 0, (25) 



Q(P) - T^ (26) 

where Nq is determined by the normalization of Q{p)- 

We need to estimate the success rate of an agent as a function of p. 
Consider first any agent i. He is characterized by his strategy ai and his 
probability Pi, that is, by the pair (a"j,pj). At each time step, he uses ctj with 
a probability Pi and 1 — ctj with a probabihty 1 — Pi- On average, this agent 
uses an effective strategy defined as 

af^ =p,ai + {l-pi)il-cji). (27) 

Switching to the components, to each 1 in cxj corresponds a Pi in a^ , while 
to each in ctj corresponds a 1 — pj in a^. Consider now two agents i 
and j characterized by pairs {(Ji,pi) and {aj,pj) respectively. Comparing the 
effective strategies of these two agents component by component, there are 
4 possible different realisations of the components, as presented in table 2. 
In this table, {c^i )k is component k of the effective strategy of agent i. If 
d is the distance between ai and ctj, c?e//, the distance between the effective 
strategies of agents i and j, is 

4// = d\l-{p,+pj)\ + {l-d)\p,-pj\. (28) 

We call deff the effective distance between {ai,pi) and {aj,pj). To find the 
average effective distance between agent i and the other agents, two averaging 
processes must be performed. The first one is over d, using the distance 
distribution P{d), and gives 
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{deff)p(d) = 2(11 - {P^+Pj)\ + \P^-PJ\)■ (29) 

The second average is over the probabihty distribution Q{pj) and gives 

deffip) = -l dp,Q{p,){\l - {p + Pj)\ + \p-Pj\) (30) 

where we drop the subscript i from now on. The success rate as a function 
of p is then given by 

N/2-3/2 (Tvr_-i\] 

<p)= E ^!(ir3Tz-)y4r(i-^^//r (31) 

using Eq. ([TT|). 

Equations (|2B|), ( PPD and (^TJ) completely determine r(p) and Q{p). To 
approximate both these quantities, we assume that Q{p) has a uniform dis- 
tribution, that is Q{p) = 1 for p between and 1. Eq. ( ^Of ) gives 



defAp) = \{l-2p + 2p'). (32) 

The average effective distance is a maximum at p = and 1 where it is equal 
to 0.5. Otherwise, d^jf < 0.5, achieves its minimum value 0.25 at p = 0.5. 
Consequently, the success rate of the agents for p around or 1 is higher 
than for p around 0.5. Iterating equations (^61) , (^0|) and (^) allows us to 



make successive approximations to Q{p)- An approximate solution for NQ{p) 
after a hundred iterations is presented in Fig. 2 with the results of a direct 
numerical simulation of the system. We repeated our iterations starting 
with a Gaussian distribution, an asymmetric distribution and a parabolic 
distribution for Q{p). These all converged to the same result for the set of 
parameters presented at Fig. 2. In fact, only 2 or 3 iterations were enough to 
reach a stationary distribution. As Fig. 2 illustrates, the agreement between 
the numerical iteration of the analytical results and the direct numerical 
simulation of the model is very good. 

It is interesting to note that most of the agents are associated with a 
probability near 1 or 0. Such agents use only one strategy most of the time, 
either their own or the opposite one, respectively. This conclusion compares 
with the conclusion of the previous section. In fact, it was shown that if the 
agents are allowed to choose between strategies they do not do as well as 
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if they were forced to use only one strategy permanently. In the adaptive 
model of the previous section, the agents were not able to recognize this 
fact and performed badly. In the evolutionary model we investigated in this 
section, the agents are able to recognize this and most of them evolved to a 
state where they were using only one strategy. Hence, a generalization of this 
evolutionary model seems to be the solution of the problem of the adaptive 
model. 

5 Conclusions 

We have shown how the Hamming distance between strategies is a useful 
tool to investigate the properties of the minority game. It is a measure of the 
difference between two strategies and corresponds to a time average of the 
different predictions of these two strategies, assuming that all the histories are 
likely to occur. We successfully applied this notion to compute the average 
success rate of the agents in different versions of the minority game. For every 
version, the analytical results were complemented by appropriate numerical 
simulations. 

In an adaptive version of the model, the difference between the success 
rates of the used and unused strategies was emphasized. It was shown that 
the adaptation process is ineffective, increasing the success rate of the unused 
strategies instead of the success rate of the strategies used by the agents. In 
fact, if the agents were forced to use one strategy permanently, they would 
do better. To illustrate this conclusion, the adaptive model was compared 
to a stochastic version of the minority game. It was shown that adaptive 
agents having a set of s strategies at their disposal perform worst than agents 
taking decisions according to a strategy chosen a random at each time step 
among the same set of strategies. Our analytical results suggest that the 
problem with the adaptive process is that every agent is using the same fixed 
mechanism to adapt. As an improvement, we suggest it would be interesting 
to let the method used by agents to select strategies evolve. Consequently, 
a version of the minority game in which the agents strategies evolve was 
investigated. 

In the evolving version of the model, the agents have only one strategy, 
and its opposite, at their disposal. It was shown numerically and analyti- 
cally that most of the agents spontaneously use only one of these strategies. 
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Hence, in this version of the minority game as well as in the previous adaptive 
version, an agent benefits from always using the same strategy. In the adap- 
tive version of the game, agents do not realize this, in the evolving version, 
the agents evolve to this state. We proposed that a simple generalization of 
this model will improve the efficiency of the adaptive process. 

As a corollary, let us mention the work of Cavagna [0 , who claims that the 
only relevant parameter for the memory is the value of m, not the information 
contained in the memory. In fact, all our results are obtained using only 
geometrical considerations, supporting this conclusion. 



14 



References 

[1] See for instance C Sinclair, Introduction to Economics, 6th ed. (Butter- 
worth, London, 1982). 

[2] N. F. Johnson, P. M. Hui, R. Jonson and T. S. Lo, Phys. Rev. Lett. 82, 
(1999) 3360 ( |cond-mat/9810142|) . 



[3] W. B. Arthur, Am. Econ. Assoc. Papers and Proc. 84 (1994) 406. 



[4] D. Challet and Y.-C. Zhang, Physica A 246 (1997) 407 (^Aa^- 
org/970800BO. 



[5] D. Challet and Y.-C. Zhang, Physica A 256 (1998) 514 (|3nd= 
mat/980508l . 



[6] N. F. Johnson, M. Hart and P. M. Hui, |cond-mat/9811227| 16 Nov 1998. 



[7] A. Cavagna, Phys. Rev. E 59, (1999) R3783 ( |cond-mat/9812215|) 



15 



Table 1 



Table 2 



data 


a 


a' 


000 


1 





001 


1 


1 


010 








Oil 





1 


100 


1 


1 


101 


1 


1 


110 





1 


111 









(af^h 


i^fh 


\{<yr%-{<y-")u 


Pi 


Pj 


\pi - Pj 


1-p. 


Pj 


\^-{pi+Po) 


Pi 


1-p, 


\^-{pi+P]) 


l-p^ 


l-p, 


\pi - Pi 



16 



Table Captions 



Table 1 — The first column lists all the possible histories of the system for 
the last three time steps {m = 3). A strategy is a set of decisions for all the 
different possible histories. Two example strategies a and a' are shown in 
the second and third columns. 



Table 2 — The first two columns list the 4 possible realisations of the 
components of the effective strategies of two agents characterized by pairs 
{(Ji,Pi) and {aj,pj). {(J^ )k is component k of the effective strategy of agent 
i. The last column gives the different possible contributions to the calculation 
of the effective distance. 
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Figure Captions 



Figure 1 — Analytical results (line) and numerical simulations (line and 
crosses) for the success rates of the agents and their strategy for (a) the 
stochastic and (b) the deterministic versions of the model; the two upper 
curves are the success rates of the strategies and the two lower curves are the 
success rates of the agents, for both (a) and (b). The choice of the parameters 
is A^ = 1001 and m = 5. 40 simulations were done over 500 time steps. 



Figure 2 — Comparison of the analytical (line) and numerical (line and 
crosses) results for NQ{p), the average number of agents with a probability 
p, for A^ = 101, m = 3, L = —4 and R = 0.2. The results of the direct 
numerical simulation are obtained after 200 simulations of 10^ time steps. 
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